Field
Embodiments presented herein generally relate to self-maintenance background processes executing in a distributed file system, and more specifically, to adaptive techniques for scheduling such background processes.
Description of the Related Art
Distributed file systems often rely on background processes that update and correct inconsistencies on the file system. For example, a file system may use journaling to write data to disk. In such an approach, the file system initially writes the data into a journal to achieve relatively fast write I/O. And a maintenance process executes in the background to ensure that the data is eventually written to disk, e.g., by flushing the data in the journal to disk and updating pointers to the data. As another example, the file system may handle delete operations on stored data by removing pointers to the data. Although other blocks of data no longer reference the data (and the blocks in which the data was stored are indicated as free), the data remains stored on disk. A background process may later perform garbage collection to actually remove the data from the blocks so that the file system may use the blocks to store new data. A distributed file system may constantly employ a number of self-maintenance background processes to ensure that the file system remains consistent.
However, many self-maintenance processes are resource intensive. For example, some processes involve large data I/O, memory, and execution time (e.g., due to scanning the entire file system, performing large I/O operations, etc.). Although the processes execute in the background, the processes may nevertheless affect performance of foreground processes that generally take greater precedence, such as user-initiated file system activities. As a result, system administrators aim to schedule self-maintenance processes to execute during periods where minimally interfere with such foreground processes. For instance, an administrator may schedule a self-maintenance process relative to time, e.g., schedule a garbage collection process to execute at a specified time. An administrator may also specify that a process be executed in response to a given event, e.g., if a user deletes a file exceeding a given amount of disk space. And an administrator may also schedule a process to occur in response to a system state, e.g., if an amount of free disk space is below a specified threshold.